Network data storage is most economically provided by an array of low-cost disk drives integrated with a large semiconductor cache memory. A number of data mover computers are used to interface the cached disk array to the network. The data mover computers perform file locking management and mapping of the network files to logical block addresses of storage in the cached disk array, and move data between network clients and the storage in the cached disk array. See, for example, Vahalia et al. U.S. Pat. No. 5,893,140 issued Apr. 6, 1999, entitled “File Server Having a File System Cache and Protocol for Truly Safe Asynchronous Writes,” incorporated herein by reference.
Typically the logical block addresses of storage are subdivided into logical volumes. Each logical volume is mapped to the physical storage using a respective striping and redundancy scheme. The data mover computers typically use the Network File System (NFS) protocol to receive file access commands from clients using the UNIX (Trademark) operating system or the LINUX (Trademark) operating system, and the data mover computers use the Common Internet File System (CIFS) protocol to receive file access commands from clients using the MicroSoft (MS) WINDOWS (Trademark) operating system. The NFS protocol is described in “NFS: Network File System Protocol Specification,” Network Working Group, Request for Comments: 1094, Sun Microsystems, Inc., Santa Clara, Calif., March 1989, 27 pages, and in S. Shepler et al., “Network File System (NFS) Version 4 Protocol,” Network Working Group, Request for Comments: 3530, The Internet Society, Reston, Va., April 2003, 262 pages. The CIFS protocol is described in Paul J. Leach and Dilip C. Naik, “A Common Internet File System (CIFS/1.0) Protocol,” Network Working Group, Internet Engineering Task Force, The Internet Society, Reston, Va., Dec. 19, 1997, 121 pages.
The data mover computers may also be programmed to provide clients with network block services in accordance with the Internet Small Computer Systems Interface (iSCSI) protocol, also known as SCSI over IP. The iSCSI protocol is described in J. Satran et al., “Internet Small Computer Systems Interface (iSCSI),” Network Working Group, Request for Comments: 3720, The Internet Society, Reston, Va., April 2004, 240 pages. The data mover computers use a network block services protocol in a configuration process in order to export to the clients logical volumes of network attached storage, which become local pseudo-disk instances. See, for example, Jiang et al., Patent Application Publication US 2004/0059822 A1 published Mar. 25, 2004, entitled “Network Block Services for Client Access of Network-Attached Storage in an IP Network,” incorporated herein by reference.
A storage object such as a virtual disk drive or a raw logical volume can be contained in a file compatible with the UNIX (Trademark) operating system so that the storage object can be exported using the NFS or CIFS protocol and shared among the clients. In this case, the storage object can be replicated and backed up using conventional file replication and backup facilities without disruption of client access to the storage object. See, for example, Liang et al., Patent Application Publication US 2005/0044162 A1 published Feb. 24, 2005, entitled “Multi-Protocol Sharable Virtual Storage Objects,” incorporated herein by reference. The container file can be a sparse file. As data is written to a sparse file, the size of the file can grow up to a pre-specified maximum number of blocks, and the maximum block size can then be extended by moving the end-of-file (eof). See, for example, Bixby et al., Patent Application Publication US 2005/0065986 A1 published Mar. 24, 2005, entitled “Maintenance of a File Version Set Including Read-Only and Read-Write Snapshot Copies of a Production File,” incorporated herein by reference, and Mullick et al., Patent Application Publication 2005/0066095 A1 published Mar. 24, 2005, entitled “Multi-Threaded Write Interface and Methods for Increasing the Single File Read and Write Throughput of a File Server,” incorporated herein by reference.
The storage technology described above, in combination with a continuing increase in disk drive storage density, file server processing power, and network bandwidth at decreasing cost, has provided network clients with more than an adequate supply of network storage capacity at affordable prices. The cost of the network file server and its attached storage, however, is becoming a small fraction of the total cost of ensuring fast and reliable access to a vast and ever increasing mass of stored information. The total cost is becoming dominated by the cost of administering the mass of stored information over its lifetime, including the cost of software tools for hierarchical storage, data backup, remote replication, and other kinds of information lifecycle management (ILM). See, for example, Amegadzie et al, Patent Application Publication US 2006/0212746 published Sep. 21, 2006, entitled “Selection of Migration Methods Including Partial Read Restore in Distributed Storage Management,” incorporated herein by reference. The vast amount of stored information is also interfering with quick recovery from hardware or software errors that require storage to be taken offline for a file system consistency check.